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at Cerritos College, California. The author contends that, although some 
innovations work and others do not, the evaluation process is in itself an 
innovation and is essential to the innovation development cycle. 

Understanding and reporting program outcomes subsequently advances the 
development and acceptance of theory and practice. The author's evaluation of 
the LCP began with the specification of initial questions designed to address 
the following four program objectives: (1) expand and design student 

activities and services to increase student retention, persistence, and 
success; (2) increase student involvement with activities and services; (3) 
improve faculty involvement with students and learning communities; and (4) 
increase campus commitment to learning communities. The questions were open- 
ended. The evaluation was conducted in spring 2001; a total of 531 students 
were enrolled in learning communities in 2000-2001. Noteworthy findings of 
the evaluation are. as follows: (1) 90% of the students in the LCP were from 

underrepresented groups; (2) the majority of students in LCP intended to 
transfer to earn a B.A. degree; (3) success rates in the learning communities 
were better than comparable courses in more than half of the 48 cases 
observed. (Author/NB) 
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Preface 



Learning Communities Program Description 

Cerritos College 

Overview 



Learning Communities involve faculty working together to restructure curriculum. The 
intent behind such restructuring is to help students: 

(1) make intellectual connections between two subjects 

(2) make connections with each other 

(3) make connections with the campus 

(4) make connections with their instructors 

The intents are grounded in retention theory, which reveals that students who drop out of 
school are those who feel disconnected on each of these fronts. 

The criteria that characterize a learning community are: 

(1) Common cohort of students 

(2) Interdisciplinary structure 

(3) Theme based 

(4) Discipline integration via aligned lectures, mutual assignments, aligned course 
objectives, etc. 

Features 



• Students volunteer to join this program. 

• All those who volunteer for the program are accepted, provided they fulfill 
prerequisite requirements. (These requirements vary from course to course.) 

• The number of students participating in a team course depends on the course cap. 
For example. Speech courses have a cap of 25-30 students, yet Political Science 
courses often have a cap of 60 students. If two such courses are paired, the 
department deans and instructors negotiate the learning communities cap. As an 
example, a speech class might be willing to accept 40 students instead of 30, 
while a Political Science class might be willing to bring the number of students 
down from 60 to 40. 

Learning Communities courses at Cerritos College are taught in one of two ways: (I) 
paired (2) in teams. 

(I) Paired means courses are taught by two instructors from different disciplines 
independently, yet coordinated. Coordinated meaning the instructors work 
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together to integrate and link both their lectures and course assignments. 
Oftentimes, instructors will write a mutual syllabus and coordinate their lectures 
throughout the semester to demonstrate to the students how the two disciplines 
(i.e. courses) are related to each other. In addition, each instructor would have the 
same students at different times, and the courses are typically scheduled back-to- 
back (e.g.. Speech 9:00-10:00, Political Science 10:00-11:00). 

(2) Team-taught courses are those in which both instructors are in the class at the 
same time teaching the same group of students through both class meeting times 
(e.g. Speech 9:00-10:00, Political Science 10:00-11:00, with both instructors and 
students in the class together from 9:00-11:00.) 
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Learning Communities Program Evaluation 
Cerritos College 
Spring 2001 

by 

Jan Connal, Ph.D. 



Introduction 



Like many institutions of higher education, Cerritos College maintains a culture of 
instructional innovation. Faculty take great pride in the academic responsibility for 
instructional innovation. Faculty are encouraged and expected to develop 
programs, services and professional skills. The institution supports instructional 
innovation with limited professional and program development funds. Substantial 
development, however, is usually funded by categorical grants that require more 
formal evaluation and accountability. 

As with any creative endeavor, some innovations “work” while others do not. At 
times the innovation development process may appear to critics as wasting time 
and money. This would be true if the development efforts were not evaluated to 
determine positive outcomes. Unfortunately, all too few innovations are credibly 
evaluated and reported. 

It is my assertion that evaluation is itself an innovation and essential to the 
innovation development cycle. Genuine innovation does not happen apart from 
evaluation. Unfortunately, faculty are more often consumed with the development 
and application of an innovation, rather than the formal evaluation of it. 
Consequently, faculty risk missing the full promise of instructional innovation and 
leave critics’ claims unanswered. 

Evaluation, if done well, leads to a more complete understanding of the effects of 
innovation. Understanding and reporting program outcomes subsequently 
advances the development and acceptance of theory and practice. 
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Program Evaluation Methodologies 

A variety of methods can be used to evaluate program outcomes. Much has been 
written elsewhere about different models of program evaluation. For a thorough 
treatment of this topic I refer you to, among others, Educational Evaluation: 
Theory and Practice by Worthen and Sanders. As a means of introducing the 
Learning Communities Program evaluation at Cerritos College, however, I will 
briefly review two broad categories of methods and measurements: quantitative 
and qualitative. 

The most commonly used methods in program evaluation are those that generate 
quantitative data like student success and retention rates. Program evaluations 
often employ surveys of students and faculty to assess satisfaction with the 
program. Some surveys address more behavioral measures like how frequently 
students visit the instructor during office hours. In general, quantitative data is 
favored and accepted as being more characteristic of objective, concrete research. 

The generation of qualitative data, on the other hand, is rare in program evaluation. 
Several reasons exist for this paucity. The majority of educational researchers are 
trained in quantitative methods. This is true as well for the educational 
administrators making decisions about programs. Researchers and decision- 
makers, alike, generally favor using methods they know best. The generation of 
qualitative data requires more time and effort, and hence more money. Conducting 
focus groups, interviews and field studies certainly yield a depth of information but 
at a high cost. And conclusions drawn from qualitative research are often criticized 
as being too subjective, unlike concrete quantitative research. 

Although it may seem that quantitative and qualitative methods are incompatible, I 
propose along with other practioners that the two methods can and should be 
combined. Jackson’s (1999) evolutionary research model combines the methods in 
a series of iterative processes. This model assumes 1) research is ongoing, that 
theory development is continuous; 2) there is need to think both deductively and 
inductively at any given point in time, that the evaluation process goes back and 
forth; 3) the research question(s) determines which approach is appropriate; 4) 
subsequent questions and direction may necessitate a new method; 5) quantitative 
and qualitative research should provide a convergence of findings; and 6) by 
combining qualitative and quantitative methods the researcher should find 
complimentary methodological solutions. 
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It is certainly more complex to develop and conduct an innovative evaluation such 
as suggested by Jackson’s model above. But to fully understand and assess the 
outcomes of an instructional innovation like the Cerritos College Learning 
Communities Program, nothing short of an evaluation innovation would suffice. 



Cerritos College Learning Communities Program Evaluation 

The evaluation of the Learning Co mm unities Program at Cerritos College has 
evolved over time. And, as a result, much has been learned about instructional 
innovation, student learning and evaluation itself. A fundamental lesson learned 
was that it is not enough to know what worked, we must understand why the 
innovation works. The purpose of this paper is to share the Cerritos College 
evaluation experience. Hopefully, this experience will inspire others to view 
evaluation as an innovation to more fully understand how and why learning 
communities work (or don’t work). 

The evaluation of the Learning Co mmu nities Program began with the specification 
of initial questions. The questions were generated to address four program 
objectives. 

Objective 1: Expand and design student activities and services to increase 
student retention, persistence and success. 

1 . Which activities or services were expanded and developed? 

2. How many students were informed? 

3. How many students used the services and activities? 

4. Were students who used the services more successful than those who didn’t? 

5. Did students who used services acquire more knowledge about 

a. Commitment to education? 

b. Clear educational purpose? 

c. Personal responsibility? 

d. Time management? 

e. Goal setting? 

f. Defining purpose? 

g. Understanding consequences of probation? 

6. Did students use more services? 

7. Were students more informed of services available? 

8. Were contacts made with first-time students? Did they use services offered? 

9. Were contacts made with probation students? Did they use services offered? 
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Objective 2; Increase student involvement with activities and services. 

1. Was a new semester celebration held? How many students attended? 

2. Were faculty-student mixers held? How many were involved? 

3. Did participating students feel more able to interact with instructors? 

4. Did instructors tell students to use services provided to them? 

5. Did instructors speak to students outside of class? 

6. Were learning activities provided which increased student self-awareness 
and commitment? 

7. Did the activities foster student involvement and commitment? 

8. Were class visits made? By whom? Which classes? 

Objective 3: Improve faculty involvement with students and learning 
communities. 

1. Were orientation sessions held for faculty? 

2. How many attended? 

3. What did faculty learn from the orientation? 

4. Were faculty who attended more involved as a result? 

Objective 4: Increase campus commitment to Learning Communities. 

1. Were monthly workshops held to highlight student support services? 

2. Were classroom presentations made? 

3. Did faculty encourage students to use services? 

4. Did students get an orientation to the Library? 

5. Did students get an orientation to the LAP? 

6. Did students use these services as a result of the orientations? 

Upon discussing how to proceed with the evaluation it was agreed to move away 
from the initial questions, those that could be answered ‘yes’ or ‘no,’ in favor of 
why and how questions. Prior to the current evaluation retention rates, success 
rates and satisfaction surveys had been routinely collected and compared. 

Program staff knew the status of the program but needed to know more in order to 
make program improvements. It was agreed that a more full understanding of the 
program was needed and a fresh approach was in order. The time was right to 
introduce the concept of a progressive evaluation innovation. 

The current evaluation was conducted Spring, 2001 using a combination of 
quantitative and qualitative methodologies. The multi-method approach was used 
to better understand the true effects of Learning Communities on students and 
faculty. The current evaluation was particularly interested in the impact of student 
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support and guidance provided to the learning communities. The following 
information gathering techniques were included in the multi-method evaluation: 

1. Success rates of students from Learning Conununity (LC) courses and 
comparable Non-Learning Conununity (NLC) courses. 

2. Student surveys of LC and NLC course drop-outs. 

3. Student surveys in LC’s, comparing the course beginning (Time 1) to the 
end of term (Time 2). 

4. LC student focus groups. 

5. LC student and faculty interviews. 

6. LC and NLC classroom observations and ethnographies. 

7. LC Retention Counselor interview. 

The Learning Conununities Program at Cerritos College included 1 1 combinations 
in Fall 2000 and another 12 in Spring 2001. A total of 531 students were enrolled 
in Learning Conununities in 2000-2001. Nearly 20 retention and academic 
support workshops were available to students in the Program. 

The evaluation project was designed and coordinated by Jan Connal, Ph.D. The 
team of researchers involved in the multi-method evaluation included Cerritos 
College colleagues Danielle Carney, Tanisha Peoples, Mary Kay Toumajian, Dora 
Lee, Karen Pinaud and Steve Wong. 

Noteworthy Findings 

The evaluation innovation provided a wealth of information about the Learning 
Conununities Program. The highlights are presented below. 



Who enrolled and why? 

• Ninety percent (90%) of the students in Learning 
Conununities were from underrepresented groups; 
most were first generation, under prepared students. 

• The majority of students in Learning Conununities 
intend to transfer to a university to earn a Bachelor’s 
degree. 
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• Learning Community students see themselves as 
highly motivated and goal-oriented. 

• Students initially enrolled in Learning Communities 
because the courses were transferable and seemed 
interesting; students also thought they offered 
additional support. 

• Students were more enthusiastic about enrolling in 
Learning Communities when they could understand 
the relationship between the courses. 

Success Rates 

• Success rates for the Learning Communities were 
better than comparison courses in more than half of 
the 48 cases observed. 

• In 17 out of the 23 Learning Communities studied at 
least one of the courses in the Community had better- 
than-average success rates. 

• Those who dropped the Learning Community gave 
academic difficulty as their reason more often than 
students dropping comparable courses. 

Counseling and Guidance Effectiveness 

• Learning Communities involving Counseling and 
Guidance courses with basic skills English courses 
had lower-than-average success rates (compared to 
traditional, stand-alone English courses). 

• The shortened version of the Counseling and 
Guidance course (30 minutes each week) was not well 
received. Not enough time was available for 
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establishing community and connecting the content of 
the paired courses. 

• Students gave relatively poor ratings to general 
Counseling Services; it seemed as though not enough 
counselor time was given to student needs. Students 
felt short-changed after waiting in lines. 

• Faculty did not require student involvement in 
retention services or workshops. 

• Some faculty encouraged students to attend the 
retention workshops. Most did not. Faculty received 
numerous notices about workshops but didn’t want to 
take up limited class time discussing the workshops or 
referring students. 

• Very few students (only 78) took advantage of the 
retention workshops. Learning Community students 
were more likely to use the Learning Assistance 
Program and Tutoring Services. 

• The retention counselor services were not fully 
integrated into the Learning Community Program; 
More time and dialog with the faculty was needed for 
integration to occur. 



Greater Sense of Community 

• Students in the Learning Communities felt more 
connected to each other and their instructors; this was 
true even if the student eventually dropped out of the 
Learning Community. 

• Students who persevered found the Learning 
Community to involve greater networking, 
collaboration, and teamwork than traditional courses. 
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• Learning Community students had greater interaction 
with the faculty and felt more involved, connected, 
and more comfortable asking questions. Faculty 
knew students by name which greatly enhanced 
student commitment. 

• Learning Community faculty were more caring and 
offered more guidance than faculty in comparable 
courses. 

• By the end of the Learning Community experience, 
students had moved away from their initial belief that 
success in the course depended on the instructor 
(presumably students began to take more personal 
responsibility for their success). 

• Learning Community faculty gave more 
encouragement, engaged in more dialog and 
interaction with students than did their traditional 
counterparts. 



Conclusions about the Cerritos College Learning Communities Program 

The evaluation of the Learning Communities Program at Cerritos College revealed 
a great deal about its students, faculty, program and evaluation practices. Two 
major points will serve as my conclusion: the mixed success of the Program and 
the lack of integration of retention services. 

Mixed Success within the Program. The Learning Communities Program is clearly 
a success in establishing a sense of community among its participants. Learning 
Community course success rates, however, were only slightly higher than 
comparable courses. 

Why wouldn’t course success rates be higher? A possible answer may lie in the 
differences between student and faculty expectations. Given the fact that students 
who enrolled in Learning Communities courses were more often under prepared, 
generation college students, it is possible that, despite their high motivation and 
aspirations, they lacked the role models and understanding of the educational 
system and expectations necessary for success. Many enrolled in the Learning 
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Community because courses were transferable (reflecting high aspirations) and 
appeared to offer more support than traditional courses (reflecting a desire/need for 
additional help). Couple this with the fact that Learning Communities faculty 
worked at creating embellished curriculum and activities which may have resulted 
in rigorous academic standards well beyond the reach of many of the students. In 
fact, a fair number of students reported not seeing the connection between the 
courses in the Learning Community and those who dropped out gave reasons of 
academic difficulty. Better monitoring of the expectations of students and their the 
mastery of connected course content may lead to greater success in the Learning 
Communities. 

Lack of Integration of Retention Services. There also appeared to be a mismatch 
between students’ need for and use of retention services. Given the fact that 
Learning Communities students are often under prepared and lack an 
understanding about succeeding in college, it would appear students need and want 
additional help to succeed in the Program. Very few students, however, attended 
the workshops provided. It was not for a lack of effort on the part of the Retention 
Counselor, however. Although faculty were inundated with flyers, e-mails, and 
voice-mail, relatively few made time to discuss the services with students. Fewer 
still required student participation in the workshops. According to the Retention 
Counselor, retention services were seen as “add-ons” and not integral to the 
Learning Communities Program. Perhaps more needs to be done to illustrate the 
need and value of retention services. 



Recommendations for Future Evaluations 



As with any innovation there are recommendations for improvement. In the 
context of evaluating learning communities, I have the following to offer: 

1. Obtain an early commitment from Learning Community faculty to 
participate in qualitative information gathering efforts. The evaluation is 
critical to improving the program. 

2. Convince Learning Community faculty to use weekly classroom assessment 
techniques to monitor the instructional process and the formation of 
‘community.’ 
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3. Convince Learning Community faculty to require their students to develop 
learning portfolios which can later be used to assess student learning along 
with course grades. 

4. Evaluate the program after it is known to be sufficiently well implemented. 

5. Evaluate the program over time, following up on students to determine 
lasting effects. 
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